Customer Value Management with Transaction Data

Introduction

In this lab, you are going to practice customer value management using R.

More specifically, you will learn how to:

Analyze Transaction Data
- Descriptive Statistics
- Data Visualization
Customer Segmentation And Labeling
- Customer Clustering
- Group Attribute Analysis (Static Attributes)
- Group Dynamic Analysis (Development Trends)
- Inter-group Flow Probability

From these analyses, we can see the company’s main sources of revenue and profit. We can also see whether these profit-generating groups are showing trends of growth or decline. Based on this, we can set marketing priorities, decide on marketing strategies, and plan marketing tools. In addition to the aforementioned descriptive statistics, cluster analysis, and data visualization, we can also use these simple transaction records to achieve:

Predictive Model Learning
- Retention Probability
- Expected Revenue
- Probability of Switching between Groups
- Next Likely Purchase Time

Using these predictions, we can carry out fully customized:

Customer Value Management:
- Customer Lifetime Value
- Customer Acquisition Strategy
- Customer Development Strategy
- Customer Retention Strategy
Targeted Marketing:
- Designing Marketing Programs
- Choosing Marketing Programs
- Selecting Marketing Targets

This material is based on Dr. Tony Chuo’s CVM example (https://bap2.cm.nsysu.edu.tw/?page_id=869). The modifications I have made include updating the packages, providing more in-depth analysis, and enhancing the comments.

Environment Setup

packages = c(
  "dplyr","ggplot2","caTools","ROCR","d3heatmap",
  "googleVis","devtools","plotly")
existing = as.character(installed.packages()[,1])
for(pkg in packages[!(packages %in% existing)]) 
  install.packages(pkg)
if(!is.element("chorddiag", existing))
  devtools::install_github("mattflor/chorddiag")
if (!require("devtools")) install.packages("devtools")

## Loading required package: devtools

## Loading required package: usethis

devtools::install_github("talgalili/d3heatmap")

## Skipping install of 'd3heatmap' from a github remote, the SHA1 (c8a3c64a) has not changed since last install.
##   Use `force = TRUE` to force installation

options(digits=4)
library(dplyr)

## 
## Attaching package: 'dplyr'

## The following objects are masked from 'package:stats':
## 
##     filter, lag

## The following objects are masked from 'package:base':
## 
##     intersect, setdiff, setequal, union

library(ggplot2)
library(caTools)
library(ROCR)
library(d3heatmap)

## 
## ======================
## Welcome to d3heatmap version 0.9.0
## 
## Type citation('d3heatmap') for how to cite the package.
## Type ?d3heatmap for the main documentation.
## 
## The github page is: https://github.com/talgalili/d3heatmap/
## Please submit your suggestions and bug-reports at: https://github.com/talgalili/d3heatmap/issues
## You may ask questions at stackoverflow, use the r and d3heatmap tags: 
##   https://stackoverflow.com/questions/tagged/d3heatmap
## ======================

## 
## Attaching package: 'd3heatmap'

## The following objects are masked from 'package:base':
## 
##     print, save

library(googleVis)

## 
## Welcome to googleVis version 0.7.1
## 
## Please read Google's Terms of Use
## before you start using the package:
## https://developers.google.com/terms/
## 
## Note, the plot method of googleVis will by default use
## the standard browser to display its output.
## 
## See the googleVis package vignettes for more details,
## or visit https://mages.github.io/googleVis/.
## 
## To suppress this message use:
## suppressPackageStartupMessages(library(googleVis))

library(chorddiag)
library(purrr)
library(plotly)

## 
## Attaching package: 'plotly'

## The following object is masked from 'package:ggplot2':
## 
##     last_plot

## The following object is masked from 'package:stats':
## 
##     filter

## The following object is masked from 'package:graphics':
## 
##     layout

rm(list=ls(all=T))

Exploring The Dataset

Loading The Dataset And Data Preprocessing

# TODO: read the file "purchases.txt" as a table and name it "X"
# There is no data header
# The data are stored as tab-separated value
# Do not convert strings in the dataset to factor variables
X = read.table(
  'purchases.txt', header=FALSE, sep='\t', stringsAsFactors=F)

# There are three columns in this dataset:
names(X) = c('cid','amount','date')

# TODO: Convert the feature "date" to Date format
X$date = as.Date(X$date)

# TODO: Show the summary of X
summary(X)

##       cid             amount          date           
##  Min.   :    10   Min.   :   5   Min.   :2005-01-02  
##  1st Qu.: 57720   1st Qu.:  25   1st Qu.:2009-01-17  
##  Median :102440   Median :  30   Median :2011-11-23  
##  Mean   :108935   Mean   :  62   Mean   :2011-07-14  
##  3rd Qu.:160525   3rd Qu.:  60   3rd Qu.:2013-12-29  
##  Max.   :264200   Max.   :4500   Max.   :2015-12-31

# Distinct number of Customers
n_distinct(X$cid)

## [1] 18417

Customer Data

# TODO: Explain what this piece of code does
A = X %>% 
  mutate(days = as.integer(as.Date("2016-01-01") - date)) %>% 
  group_by(cid) %>% summarise(
    recent = min(days),     # Number of days since most recent purchase
    freq = n(),             # Frequency of purchases
    money = mean(amount),   # Average money spent in each purchase
    senior = max(days),     # Number of days since first purchase
    since = min(date)       # Date of first purchase
  ) %>% data.frame

# TODO: Show the summary of A
summary(A)

##       cid             recent          freq           money          senior    
##  Min.   :    10   Min.   :   1   Min.   : 1.00   Min.   :   5   Min.   :   1  
##  1st Qu.: 81990   1st Qu.: 244   1st Qu.: 1.00   1st Qu.:  22   1st Qu.: 988  
##  Median :136430   Median :1070   Median : 2.00   Median :  30   Median :2087  
##  Mean   :137574   Mean   :1253   Mean   : 2.78   Mean   :  58   Mean   :1984  
##  3rd Qu.:195100   3rd Qu.:2130   3rd Qu.: 3.00   3rd Qu.:  50   3rd Qu.:2992  
##  Max.   :264200   Max.   :4014   Max.   :45.00   Max.   :4500   Max.   :4016  
##      since           
##  Min.   :2005-01-02  
##  1st Qu.:2007-10-23  
##  Median :2010-04-15  
##  Mean   :2010-07-26  
##  3rd Qu.:2013-04-18  
##  Max.   :2015-12-31

In the following, we are going to compute the average purchase cycle of returning customers (K)

For all returning customers (A$freq > 1),

\[K = \frac{\sum \text{Number of days since first purchase}}{\sum \text{Frequency of purchases}}\]

# To select "cid" of customers having value of "freq" > 1, we can use the following expression:
#   A$cid[A$freq>1]

# TODO: Compute the value of K and convert it to integer
K = as.integer(sum(A$senior[A$freq>1]) / sum(A$freq[A$freq>1]))

Data Visualization

p0 = par(cex=0.8, mfrow=c(2,2))

# Visualize A$recent, A$freq, log(A$money) A$senior as histograms
hist(A$recent)
hist(A$freq)
# TODO: Draw the histogram of the logarithm of A$money to the base 10
hist(log(A$money,10))
hist(A$senior)

par(p0)

Hierarchical Clustering

RFM (Recency, Frequency, Monetary) Clustering Using K-means

First of all, we need to decide how many clusters (i.e., groups of customer) we need.

Elbow Method is a technique that we use to determine the number of centres(k) to use in a k-means clustering algorithm. We iterate with different values of k starting from 1 to n (n being a hyper parameter). We plot a graph of k versus their WCSS ( sum of square of distances between the centroids and each points) value. The graph looks like an elbow and the point where it bends is chosen as the optimal value of k.

# Use only "amount" and "date" for clustering
# Calculate the total within-cluster sum of squares of K-means clustering results. This metric indicates how compact the clusters are, with lower values generally representing better clustering where data points are closer to their respective centroids.
fun <- function(k){kmeans(scale(A[,2:4]),k,iter.max=100,nstart = 100,algorithm='Lloyd')$tot.withinss}

# We will try from 1 to 10 clustering 
k.values <- 1:10

# Calculate the total within-cluster sum of squares value for each value of k (from 1 to 10)
fun_value <- map_dbl(k.values,fun)

# Draw the plot of the total within-cluster sum of squares against the number of clusters
plot(k.values,fun_value,type='b',xlab='number of clusters',ylab='total sum of squares')

# Apply K-means clustering with 4 clusters
k4 = kmeans(scale(A[,2:4]),4,iter.max = 100,nstart = 50,algorithm = 'Lloyd')

# Show the clustering result
k4

## K-means clustering with 4 clusters of sizes 127, 2466, 6322, 9502
## 
## Cluster means:
##    recent    freq    money
## 1 -0.2188 -0.1323  9.81089
## 2 -0.8414  2.0907  0.03353
## 3  1.2037 -0.4185 -0.12834
## 4 -0.5796 -0.2623 -0.05444
## 
## Clustering vector:
##     [1] 3 2 2 4 3 3 3 3 3 4 3 3 2 3 4 3 2 2 3 3 3 3 3 3 4 3 3 4 3 2 2 2 3 3 3 2
##    [37] 3 4 4 2 4 2 2 2 4 2 3 4 4 2 3 2 4 3 3 4 4 3 2 4 4 4 4 2 2 3 2 3 2 3 3 2
##    [73] 4 2 4 3 2 3 3 3 2 3 4 4 2 2 4 2 3 2 4 2 3 3 3 3 3 3 3 4 2 2 3 3 2 2 3 4
##   [109] 3 3 3 2 3 2 2 4 3 2 3 2 2 3 3 3 3 2 3 3 1 3 3 4 3 2 2 3 3 2 3 2 3 2 3 4
##   [145] 3 3 2 2 3 3 2 2 2 3 3 2 2 2 3 3 2 3 3 2 3 2 3 3 4 2 2 3 2 2 2 4 4 2 4 3
##   [181] 4 3 2 3 1 2 2 3 4 2 3 2 2 2 3 3 3 3 2 4 3 4 2 2 2 3 3 2 4 2 3 4 3 3 4 2
##   [217] 3 2 3 2 4 4 3 2 3 3 3 4 3 2 3 2 4 2 2 3 2 3 3 2 2 3 3 3 3 4 3 2 3 2 3 2
##   [253] 4 2 2 2 2 3 2 3 2 3 2 4 3 3 2 3 2 3 2 3 4 4 4 2 2 3 3 2 3 2 4 4 4 2 2 3
##   [289] 3 2 3 4 3 3 3 3 4 4 3 3 2 3 3 3 3 3 2 3 3 3 4 2 4 2 3 2 2 3 3 2 3 3 4 4
##   [325] 3 4 3 2 3 2 2 3 2 2 3 3 3 2 2 3 2 3 2 2 3 3 2 3 2 3 3 2 2 2 2 3 3 3 2 3
##   [361] 2 2 3 4 2 3 2 2 3 3 2 2 2 2 2 3 3 2 3 2 2 2 4 3 3 2 2 2 2 3 3 3 3 3 3 3
##   [397] 3 2 3 4 3 3 3 2 2 3 3 2 3 3 3 3 3 4 3 2 4 4 3 3 4 2 4 3 2 2 2 2 3 3 2 2
##   [433] 2 2 3 4 4 3 3 3 3 3 2 2 2 2 2 2 2 2 3 2 2 2 2 2 2 2 2 3 3 2 3 2 4 2 2 2
##   [469] 2 2 2 2 2 4 2 2 2 2 3 2 3 3 2 2 3 3 2 3 3 4 3 3 3 4 2 2 3 3 3 3 2 3 2 2
##   [505] 2 2 4 2 3 2 2 3 2 2 2 2 2 4 2 2 2 2 2 2 3 3 2 2 3 2 2 3 2 3 2 2 3 3 2 2
##   [541] 3 4 2 2 2 2 2 3 3 2 2 4 3 4 4 2 3 2 3 2 3 3 2 2 2 3 4 2 2 2 2 3 3 2 2 3
##   [577] 3 4 2 3 4 2 2 2 2 3 2 2 2 3 3 3 2 2 2 2 3 2 2 3 2 2 2 4 4 2 3 3 2 2 2 3
##   [613] 2 2 2 3 3 2 1 2 3 2 3 2 3 3 2 3 3 3 4 2 2 3 2 2 2 2 3 3 3 2 4 3 2 3 3 2
##   [649] 2 3 3 3 2 3 2 2 2 2 3 3 3 2 2 2 3 2 2 3 1 3 4 3 2 2 2 3 2 2 2 3 2 3 2 2
##   [685] 2 4 4 3 4 4 2 2 3 2 2 3 2 3 4 2 2 2 3 2 2 2 2 2 3 2 3 1 2 2 2 2 2 2 2 2
##   [721] 3 3 3 2 2 3 2 2 2 4 2 2 2 2 1 2 2 3 2 3 2 2 3 2 2 3 2 2 3 2 2 2 2 3 2 2
##   [757] 2 2 2 2 2 3 3 2 3 2 2 2 4 2 4 2 2 3 2 3 3 3 3 2 2 2 2 3 4 2 3 4 2 2 3 2
##   [793] 2 2 2 3 3 3 3 2 2 2 2 2 3 3 4 2 2 3 3 3 2 3 4 3 2 3 2 3 2 3 3 3 2 3 2 3
##   [829] 3 3 2 2 3 3 2 3 3 3 3 4 2 2 2 2 2 3 4 2 3 2 2 4 3 4 2 4 2 2 2 3 3 3 3 3
##   [865] 3 3 3 4 3 3 3 3 3 2 2 3 2 4 2 2 2 2 4 3 3 3 3 3 3 3 2 3 2 3 3 3 2 3 3 3
##   [901] 4 2 4 2 3 4 2 3 3 4 2 2 3 2 3 2 3 2 2 4 3 2 3 3 3 3 2 2 2 3 3 2 3 3 2 2
##   [937] 2 4 3 3 3 2 3 3 3 3 4 3 2 2 3 3 2 3 3 2 3 2 4 2 2 3 3 3 4 4 3 3 2 2 2 2
##   [973] 3 3 4 2 3 3 2 3 3 3 3 3 2 3 2 3 2 3 3 2 2 4 3 3 3 3 4 3 3 2 2 2 3 2 3 2
##  [1009] 3 3 3 3 2 3 1 2 3 4 2 3 2 2 2 2 3 3 2 3 3 4 3 3 3 2 3 3 3 2 3 3 3 2 2 3
##  [1045] 3 4 3 2 3 4 4 3 3 2 2 2 3 3 2 3 3 3 2 3 3 2 4 3 3 4 2 2 2 4 2 2 2 2 2 4
##  [1081] 4 3 3 4 3 3 3 3 3 3 3 4 3 3 2 2 3 3 2 3 2 2 4 4 3 3 2 2 2 2 4 2 2 2 3 3
##  [1117] 4 3 3 3 2 3 2 3 2 3 3 2 3 3 2 4 3 4 3 3 2 3 3 2 4 3 4 3 2 4 3 2 2 3 3 3
##  [1153] 3 2 2 3 3 3 2 2 4 2 2 3 3 2 3 3 3 3 3 2 3 2 2 2 3 4 2 4 2 3 3 2 3 4 4 3
##  [1189] 3 3 2 3 3 3 3 2 2 2 3 3 4 3 2 2 1 2 2 2 3 2 3 2 3 2 3 2 3 2 4 3 3 3 3 2
##  [1225] 2 3 2 3 2 2 2 3 2 2 3 2 3 4 4 3 4 3 3 3 3 3 3 3 2 3 2 2 3 2 3 3 2 3 3 3
##  [1261] 2 3 3 4 2 2 3 4 4 2 3 3 3 3 4 2 3 4 2 3 2 3 3 4 3 2 2 3 2 2 3 2 3 3 2 2
##  [1297] 4 2 2 4 3 2 2 3 2 3 2 3 3 2 3 2 4 3 3 3 2 3 2 2 2 3 4 3 3 4 3 2 2 3 3 2
##  [1333] 4 4 2 4 4 3 3 4 3 3 3 2 4 3 2 4 3 2 3 3 4 2 3 3 3 2 2 3 3 2 3 3 4 3 3 3
##  [1369] 4 3 3 3 1 2 2 2 3 3 3 4 3 3 3 3 3 3 2 2 2 2 4 3 2 4 3 3 2 3 2 3 4 2 3 3
##  [1405] 2 3 2 2 4 2 3 3 2 4 2 3 3 4 4 3 3 2 4 3 3 3 3 2 3 3 3 2 3 3 2 4 4 3 4 3
##  [1441] 2 3 3 2 4 3 3 4 2 2 3 3 4 3 2 2 3 2 3 2 4 3 2 2 2 2 2 3 3 3 3 2 3 3 3 2
##  [1477] 3 3 3 3 4 3 3 2 4 2 4 2 3 3 3 4 3 3 2 2 2 3 3 3 2 3 3 3 3 1 2 3 3 2 3 3
##  [1513] 4 2 2 3 3 2 2 3 3 3 3 2 4 4 2 2 3 3 2 4 2 2 4 3 2 2 4 2 3 3 3 2 3 3 3 2
##  [1549] 3 2 2 2 3 3 3 3 3 2 4 2 4 1 4 3 2 4 2 3 2 3 4 2 4 3 4 3 2 4 4 2 3 4 2 3
##  [1585] 3 4 4 4 2 2 3 2 3 2 2 3 3 2 3 3 3 2 3 2 3 3 3 3 3 2 3 4 2 3 2 2 3 3 4 3
##  [1621] 3 3 2 3 4 3 3 2 2 2 3 3 4 3 4 3 3 2 2 2 4 2 3 3 3 3 3 3 3 3 3 2 3 2 4 2
##  [1657] 2 2 2 3 2 3 2 4 3 4 3 2 2 2 3 2 3 3 1 2 2 2 3 3 3 3 2 2 2 3 3 2 2 3 3 3
##  [1693] 2 2 2 3 4 2 3 3 3 2 3 3 2 2 2 2 3 3 3 2 3 3 2 3 3 1 4 3 2 3 3 3 3 2 3 3
##  [1729] 1 3 2 1 2 2 4 2 2 4 3 3 2 2 2 4 3 3 3 2 4 3 3 4 3 3 3 3 3 3 2 3 3 3 3 3
##  [1765] 3 2 2 3 3 2 2 3 2 4 2 3 4 4 2 3 2 3 3 2 4 3 3 2 3 3 4 2 3 3 3 3 4 4 3 3
##  [1801] 2 3 3 3 3 2 2 4 2 3 3 2 2 4 2 2 3 3 3 3 4 3 4 2 3 4 2 3 3 3 3 4 3 3 3 3
##  [1837] 3 3 2 3 3 2 4 2 3 3 3 3 3 3 3 3 2 2 2 3 4 3 3 2 3 3 3 2 4 3 2 2 3 3 3 3
##  [1873] 3 2 2 3 2 3 3 3 4 4 3 3 3 3 3 4 4 2 3 3 4 3 2 3 3 4 3 2 3 3 2 2 2 4 4 3
##  [1909] 2 3 2 4 2 3 4 3 3 2 4 4 4 3 3 2 4 3 3 2 3 3 3 3 2 4 3 3 3 2 3 3 4 2 4 3
##  [1945] 2 2 2 3 4 3 2 3 2 3 2 2 3 4 2 2 4 2 4 3 3 4 3 2 3 2 3 3 4 3 3 3 2 3 4 3
##  [1981] 2 2 3 3 4 2 2 3 3 2 3 2 3 4 4 2 3 2 2 4 3 3 3 3 3 3 3 3 2 3 3 3 3 4 4 4
##  [2017] 3 4 4 3 3 4 3 2 3 2 4 4 2 3 3 3 3 4 2 4 4 2 3 3 3 3 3 4 2 3 2 3 2 4 3 3
##  [2053] 2 3 2 3 3 3 3 2 3 3 3 2 2 3 3 4 2 2 2 2 4 3 3 2 3 3 3 2 2 3 3 3 3 2 3 4
##  [2089] 4 3 3 2 3 3 3 4 3 3 2 3 3 3 3 3 3 3 3 3 3 4 3 3 3 3 3 3 3 3 4 3 2 3 2 3
##  [2125] 3 3 4 2 4 3 4 3 3 2 3 3 2 3 2 2 3 4 4 3 2 3 4 4 1 3 3 3 3 3 3 3 3 3 2 2
##  [2161] 4 2 2 3 3 3 3 3 2 2 3 3 3 3 3 3 3 3 3 3 3 2 2 3 2 3 3 3 3 3 4 3 3 2 3 3
##  [2197] 4 3 3 2 3 3 2 4 3 4 4 2 3 3 3 3 3 3 3 4 2 2 3 3 4 3 3 3 3 4 3 2 3 3 4 3
##  [2233] 4 3 4 3 3 3 4 2 4 2 3 4 3 3 3 3 2 4 3 3 3 2 2 3 3 4 3 3 2 2 3 2 4 3 3 2
##  [2269] 3 2 3 3 3 3 3 2 3 2 3 2 3 2 3 2 4 3 2 2 4 3 2 3 3 3 3 3 4 3 4 4 3 3 3 2
##  [2305] 3 4 3 3 3 4 2 2 2 3 2 3 3 3 3 4 2 3 4 2 3 3 2 2 2 3 3 2 3 4 2 3 3 3 3 4
##  [2341] 2 4 2 2 3 2 3 2 3 3 2 3 3 2 2 2 3 2 3 3 3 3 2 4 3 2 3 4 2 2 4 3 3 2 4 4
##  [2377] 3 2 3 3 3 4 4 4 3 3 3 3 2 2 2 3 2 3 3 4 2 2 2 3 3 2 3 4 4 3 3 2 3 3 2 3
##  [2413] 3 3 3 3 3 3 3 2 4 2 2 3 3 2 3 2 3 3 2 3 3 4 3 3 3 3 3 2 2 3 3 2 4 4 3 2
##  [2449] 3 2 3 4 3 3 3 4 3 4 2 3 4 2 2 3 3 2 3 2 3 4 4 3 4 3 2 3 2 2 2 3 3 3 4 2
##  [2485] 3 3 2 3 2 3 3 3 3 3 2 3 3 3 4 2 3 3 4 4 3 2 3 2 3 2 2 3 3 2 3 2 4 2 3 4
##  [2521] 3 3 3 3 2 2 3 2 3 3 2 4 3 4 4 2 3 3 3 2 2 3 3 3 3 2 3 4 2 2 3 2 3 4 2 3
##  [2557] 2 2 3 3 4 3 3 3 2 2 2 3 2 2 2 3 3 3 3 3 3 4 3 3 4 2 2 3 2 2 4 2 2 2 3 3
##  [2593] 3 2 2 2 3 3 2 3 3 2 4 4 2 3 3 3 3 2 2 3 2 4 4 2 3 3 3 4 3 3 2 2 3 3 2 4
##  [2629] 3 2 4 3 3 3 3 2 3 4 3 3 3 3 3 4 4 3 3 2 2 3 3 3 4 3 3 2 2 3 3 3 4 2 4 2
##  [2665] 3 4 2 3 3 2 3 3 3 4 3 2 3 3 4 3 3 2 2 4 4 2 3 3 2 4 3 3 4 4 2 4 2 2 3 3
##  [2701] 3 2 2 2 2 3 2 3 4 3 3 3 4 2 3 3 4 3 3 2 2 3 2 3 4 4 3 3 4 3 3 3 2 4 3 3
##  [2737] 2 4 3 2 2 3 3 3 2 2 3 3 3 2 2 4 2 3 3 2 3 4 3 3 3 3 3 3 2 4 2 3 3 3 2 2
##  [2773] 2 3 3 2 2 3 4 2 3 3 4 3 3 2 3 4 2 4 2 3 4 2 2 3 3 4 3 3 2 3 3 2 2 2 2 3
##  [2809] 4 4 2 4 3 2 4 2 3 2 2 3 4 3 3 3 3 3 3 3 3 3 3 3 4 3 3 3 2 4 3 3 3 3 3 3
##  [2845] 4 2 2 3 3 4 3 3 3 3 3 2 3 2 4 4 2 3 3 4 3 3 2 3 4 3 4 2 4 3 2 3 3 3 2 4
##  [2881] 2 2 4 3 4 4 3 2 3 3 2 3 2 3 2 3 4 4 3 2 2 2 2 2 3 2 3 3 3 3 2 3 4 3 4 3
##  [2917] 3 2 3 3 2 2 3 2 2 4 3 3 3 2 3 4 3 3 2 2 3 3 3 4 3 3 2 3 2 3 4 3 3 3 2 3
##  [2953] 4 3 3 2 3 3 3 3 4 3 3 3 3 3 4 3 3 3 3 3 3 3 3 3 3 3 3 3 4 3 4 2 3 3 3 3
##  [2989] 3 3 3 4 4 3 4 4 3 3 3 3 3 4 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 4 2 3
##  [3025] 3 3 3 3 2 3 4 2 2 4 3 3 3 3 3 3 3 4 3 4 4 3 3 3 3 4 2 2 3 3 3 3 4 4 3 3
##  [3061] 3 3 3 2 2 3 2 2 3 3 3 3 4 3 3 3 2 3 3 4 2 2 3 3 2 3 3 3 3 3 3 2 3 3 2 3
##  [3097] 2 3 3 3 3 3 3 3 2 4 3 3 4 3 2 3 2 3 4 2 3 3 3 3 3 3 3 3 4 3 4 2 3 3 3 3
##  [3133] 3 3 3 3 3 3 3 3 3 4 2 2 3 3 3 3 4 2 3 3 3 3 3 3 3 3 3 3 2 3 2 4 3 3 2 4
##  [3169] 3 3 4 2 4 3 3 4 3 4 3 2 3 4 3 3 2 3 3 3 3 3 3 4 4 4 2 3 4 2 4 2 3 2 3 3
##  [3205] 3 3 2 4 2 3 2 3 3 4 3 2 4 3 3 3 2 3 3 3 2 3 3 2 3 3 4 3 2 4 3 3 3 4 4 2
##  [3241] 4 3 4 3 3 3 2 3 3 3 3 4 3 4 3 2 4 3 2 2 3 2 3 3 2 2 2 3 3 4 4 3 4 3 3 4
##  [3277] 3 4 2 4 3 2 3 2 3 3 2 4 3 2 3 3 3 3 4 2 3 4 4 4 2 3 4 4 4 3 2 3 3 3 4 3
##  [3313] 3 2 2 3 2 3 3 3 4 3 2 2 3 2 2 4 3 3 3 2 3 3 2 3 2 3 4 3 4 4 3 3 4 2 2 3
##  [3349] 3 2 2 3 3 4 2 3 2 2 3 3 3 3 2 2 3 4 3 3 3 4 2 2 2 3 3 3 2 2 4 3 4 3 3 4
##  [3385] 3 4 4 4 3 4 3 3 3 2 2 4 3 3 3 2 3 4 4 3 3 3 3 3 4 4 4 3 2 3 3 2 3 3 2 3
##  [3421] 3 3 3 4 3 3 4 3 2 3 4 3 4 3 3 4 2 3 4 3 3 3 3 4 4 3 2 3 3 2 3 3 2 3 3 3
##  [3457] 3 4 3 3 3 3 4 4 3 3 4 2 4 4 3 3 4 3 2 3 3 2 3 2 3 3 2 3 3 2 2 3 3 2 4 2
##  [3493] 3 4 3 2 3 2 2 3 4 3 4 3 2 3 3 4 3 2 4 2 3 3 2 3 2 4 3 4 3 3 3 2 4 2 3 3
##  [3529] 4 4 3 3 4 3 3 2 4 2 3 4 3 3 3 2 3 3 2 3 4 3 3 2 3 4 2 3 2 3 3 3 3 2 2 2
##  [3565] 3 4 4 4 4 4 3 3 3 3 3 3 4 4 3 3 4 2 3 3 3 4 3 3 4 3 3 4 4 4 2 3 3 4 2 3
##  [3601] 3 3 3 3 3 3 2 3 3 4 3 3 4 3 4 3 3 4 3 2 4 2 3 3 3 4 3 3 3 4 2 3 3 3 4 3
##  [3637] 4 4 2 3 4 3 2 2 2 4 3 4 3 2 3 3 2 3 3 2 2 3 3 4 2 2 3 3 3 2 3 4 3 3 4 3
##  [3673] 3 3 2 3 3 3 3 4 4 2 4 4 4 3 4 4 3 4 3 3 4 2 3 4 4 3 2 3 3 3 3 3 3 2 3 4
##  [3709] 3 3 3 2 2 3 3 2 4 3 3 3 3 3 2 3 2 3 2 3 3 2 3 3 4 3 4 3 3 3 2 3 3 3 3 3
##  [3745] 4 3 3 3 4 4 3 3 3 3 4 3 4 3 4 3 3 4 3 3 3 2 3 3 2 3 4 4 3 3 3 3 3 3 3 2
##  [3781] 3 3 3 3 3 3 3 3 4 4 3 3 3 4 3 3 4 3 3 3 3 2 3 3 3 3 2 2 3 3 4 3 2 2 2 3
##  [3817] 3 4 4 2 3 3 2 3 3 3 1 3 3 4 3 4 4 3 4 3 4 3 2 3 3 3 2 3 3 4 2 3 3 3 3 2
##  [3853] 3 3 2 3 3 4 4 2 3 3 3 3 3 4 3 2 3 3 3 4 4 3 2 2 3 3 3 4 3 3 3 3 3 4 3 4
##  [3889] 3 2 3 3 2 3 2 3 3 3 3 4 3 3 3 4 3 4 2 3 3 3 4 1 3 3 3 3 4 3 3 3 3 3 3 4
##  [3925] 3 2 4 3 4 3 3 3 4 3 4 3 3 4 3 3 3 3 3 2 3 2 2 4 3 4 3 3 4 3 4 3 3 3 3 2
##  [3961] 3 3 2 2 3 3 3 3 3 3 3 3 4 4 3 2 3 3 3 2 2 3 4 4 4 3 3 4 3 3 3 3 3 3 4 3
##  [3997] 3 3 3 4 3 3 4 3 2 4 4 3 3 4 3 3 3 3 2 3 3 3 4 3 3 4 3 4 3 3 3 3 3 3 2 3
##  [4033] 3 4 3 3 4 2 3 4 3 3 3 3 3 3 3 3 3 2 3 3 3 4 4 3 2 3 3 3 3 3 3 3 4 3 4 4
##  [4069] 3 3 3 4 3 3 3 3 3 2 3 3 3 3 4 4 3 3 2 4 4 3 3 3 3 3 3 3 4 3 3 3 3 2 3 4
##  [4105] 3 3 3 3 2 3 3 3 3 3 3 3 3 3 3 4 3 4 1 4 3 3 3 2 3 3 3 4 4 3 3 2 2 3 3 2
##  [4141] 3 3 4 4 3 3 4 3 3 3 3 4 3 2 3 3 3 2 4 3 3 3 4 2 3 3 4 3 3 3 4 3 4 4 3 1
##  [4177] 4 4 3 3 3 2 3 3 3 3 2 3 3 3 3 2 2 3 3 4 3 3 3 3 3 3 3 3 4 3 4 3 4 4 3 3
##  [4213] 3 3 3 3 3 3 3 4 3 3 3 3 3 3 4 2 4 3 3 3 3 3 3 3 2 3 4 2 3 3 3 2 2 3 2 4
##  [4249] 3 3 3 3 3 3 3 4 3 3 4 4 3 3 4 3 2 3 3 3 4 3 2 4 4 4 3 2 4 3 4 3 4 3 3 4
##  [4285] 3 3 3 3 3 3 3 3 3 3 3 3 3 3 4 3 3 2 3 3 4 3 3 4 4 3 3 4 4 3 4 4 3 3 3 3
##  [4321] 3 3 3 2 3 3 3 3 3 3 4 4 2 2 1 3 4 3 3 4 3 4 2 4 3 2 3 2 3 4 3 4 3 3 3 4
##  [4357] 4 4 3 4 4 3 3 3 3 3 3 4 3 2 3 3 3 3 3 4 3 3 3 4 2 4 4 2 2 3 3 4 3 4 3 3
##  [4393] 4 3 3 3 3 3 3 3 4 3 4 3 2 3 3 3 3 3 2 3 3 4 3 3 3 3 3 4 4 2 3 2 3 1 4 3
##  [4429] 2 3 3 3 3 3 3 3 3 3 3 4 2 3 4 3 3 3 2 3 3 3 3 2 3 2 3 3 3 3 3 3 2 3 3 3
##  [4465] 3 3 2 3 3 3 4 4 3 2 3 3 2 3 3 2 2 3 3 2 3 2 4 3 3 4 3 3 3 4 2 2 4 3 3 4
##  [4501] 4 3 3 3 3 3 2 2 3 3 2 3 4 4 3 3 3 3 2 4 3 2 2 3 2 4 3 2 3 3 3 4 2 4 2 2
##  [4537] 3 3 4 2 3 3 2 3 3 3 3 4 3 3 4 3 3 4 4 2 2 2 3 3 3 3 4 4 4 3 3 3 3 3 3 3
##  [4573] 2 3 3 2 4 2 3 3 3 2 4 2 3 4 4 2 4 3 3 3 4 4 3 2 3 4 3 2 2 4 4 3 3 3 4 3
##  [4609] 3 3 3 3 3 3 3 4 3 3 4 3 3 3 3 4 3 2 2 2 3 3 3 4 3 2 3 3 2 4 3 3 3 3 3 2
##  [4645] 3 2 4 3 3 3 4 4 2 3 3 3 3 3 4 3 3 3 4 2 2 3 4 3 3 4 3 3 3 3 3 2 3 3 2 3
##  [4681] 3 3 3 4 3 3 4 3 3 2 2 4 4 4 3 2 4 4 4 2 3 3 3 3 4 3 3 3 2 3 3 3 4 3 2 2
##  [4717] 4 3 4 3 3 3 4 3 3 3 2 3 3 2 2 3 4 3 3 3 3 2 3 2 2 3 3 4 4 2 3 3 3 3 3 3
##  [4753] 4 3 3 3 4 3 4 3 3 2 3 2 4 3 3 3 4 3 4 3 1 1 1 3 3 3 3 2 2 3 4 3 4 2 3 3
##  [4789] 4 3 3 3 2 3 2 3 4 3 2 3 3 4 3 3 2 3 4 3 3 3 3 3 3 3 3 3 3 4 3 2 3 3 3 3
##  [4825] 3 4 3 4 4 3 3 3 3 3 3 3 2 2 3 2 3 4 2 3 3 3 3 3 4 2 3 4 3 4 3 3 3 3 3 3
##  [4861] 2 3 4 3 3 3 3 2 3 2 4 3 3 3 3 3 3 3 2 3 4 3 3 3 2 3 3 3 3 3 4 3 4 3 3 3
##  [4897] 3 2 2 3 2 3 4 3 3 3 3 3 3 2 3 2 3 2 3 3 3 3 3 2 3 2 2 3 3 3 4 3 4 4 4 3
##  [4933] 3 3 3 2 3 3 4 4 3 2 4 3 3 4 3 4 4 3 3 3 2 3 3 3 2 4 3 3 3 2 2 3 4 4 3 4
##  [4969] 3 3 4 2 3 4 4 2 3 3 2 3 3 3 3 3 3 3 3 3 3 3 3 4 3 4 3 4 4 4 4 3 3 4 3 3
##  [5005] 3 3 3 3 3 3 3 4 4 3 3 3 3 3 1 3 2 4 3 2 3 3 3 3 3 2 3 3 3 2 2 3 4 3 4 3
##  [5041] 4 2 3 3 4 3 4 2 2 2 3 4 3 3 2 3 2 3 3 2 3 3 4 3 4 4 3 3 3 3 3 3 2 3 2 3
##  [5077] 2 4 3 3 2 3 3 3 3 3 3 3 3 3 3 3 2 3 3 3 4 4 3 3 2 3 3 4 3 2 2 3 3 3 3 4
##  [5113] 4 2 3 3 2 3 3 3 3 4 3 3 2 3 3 2 3 3 3 3 3 2 3 4 3 3 3 3 2 3 2 3 3 3 3 4
##  [5149] 3 4 4 4 3 3 4 2 3 3 2 3 2 4 4 4 2 3 4 3 3 2 3 4 3 2 3 4 3 3 3 3 3 3 3 2
##  [5185] 4 3 2 2 3 2 3 2 3 4 3 3 3 2 2 3 4 3 4 3 2 3 2 3 4 3 3 2 3 2 3 2 3 2 2 3
##  [5221] 3 3 4 3 2 4 3 4 4 3 3 3 4 4 2 3 3 4 3 3 3 2 3 4 2 3 4 2 2 3 3 3 2 3 3 3
##  [5257] 3 2 2 3 4 2 4 3 3 3 3 4 3 3 4 3 3 2 3 3 4 2 4 3 3 3 3 3 4 3 3 3 3 2 3 2
##  [5293] 3 3 3 2 4 2 3 3 2 4 4 3 4 3 3 3 3 2 2 3 3 2 3 4 3 4 3 3 4 4 3 3 3 2 3 2
##  [5329] 3 4 4 3 2 2 3 3 4 3 3 3 3 4 3 3 2 3 3 2 3 2 3 2 4 2 2 3 3 2 3 3 3 3 3 2
##  [5365] 3 3 2 3 2 2 2 3 3 2 4 2 2 4 2 3 3 3 3 4 4 3 2 3 2 2 2 3 2 4 3 3 2 2 3 4
##  [5401] 3 3 4 4 3 3 2 3 2 3 3 2 4 3 2 3 3 4 4 2 3 3 2 3 3 3 3 3 3 3 3 3 3 3 2 2
##  [5437] 4 4 2 2 2 3 4 4 3 3 2 2 4 3 3 4 3 3 3 3 2 3 4 2 4 2 4 4 3 3 4 4 3 3 3 3
##  [5473] 2 3 3 3 3 3 3 3 3 3 2 2 3 4 3 4 3 3 3 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 4 3
##  [5509] 3 2 3 3 3 3 3 3 3 3 4 2 3 3 3 3 4 3 3 3 3 3 3 4 3 4 3 3 3 3 3 3 3 3 2 3
##  [5545] 2 3 4 3 3 3 3 2 3 3 3 3 2 3 3 4 3 2 3 3 2 3 3 4 3 3 3 3 4 3 3 3 3 3 3 3
##  [5581] 4 3 3 3 3 3 3 3 4 3 4 4 4 4 3 3 3 2 3 4 3 3 2 3 3 2 3 3 4 3 3 2 2 3 3 3
##  [5617] 3 3 3 3 3 3 3 4 3 4 3 3 3 3 3 3 3 2 2 4 3 4 4 3 3 3 2 3 3 4 3 3 3 3 3 3
##  [5653] 3 3 2 4 3 3 3 2 4 3 3 1 3 3 3 2 3 3 3 3 3 3 3 4 4 3 3 3 2 4 2 3 3 3 2 3
##  [5689] 3 3 3 2 3 3 4 3 4 4 3 4 3 4 4 2 3 3 4 3 3 4 3 3 3 3 4 3 3 3 3 3 3 3 3 3
##  [5725] 3 2 4 3 2 3 3 4 3 3 3 3 3 2 2 2 2 3 2 2 2 3 2 2 4 4 2 3 2 4 3 4 3 2 3 3
##  [5761] 3 3 3 3 3 2 3 3 2 3 3 3 4 3 4 4 3 2 3 3 3 2 3 4 3 4 3 3 4 4 3 3 3 3 2 3
##  [5797] 3 3 3 2 3 3 4 4 3 3 2 3 2 2 3 3 3 3 4 3 3 4 2 3 3 3 3 3 2 3 2 2 4 3 2 3
##  [5833] 2 2 3 3 4 2 3 3 3 3 3 4 3 3 3 2 3 3 3 3 3 2 3 3 2 2 3 4 2 2 3 3 3 3 3 2
##  [5869] 3 2 3 4 2 3 3 3 3 2 2 2 2 3 3 4 3 2 3 3 2 4 4 2 2 2 4 3 3 3 3 3 3 4 3 3
##  [5905] 4 3 4 2 3 3 3 3 3 3 3 3 3 3 3 3 3 2 3 4 3 3 3 2 2 3 4 3 4 3 2 3 3 3 3 3
##  [5941] 2 3 4 4 3 2 3 2 3 3 3 3 3 4 2 3 4 3 3 2 3 3 3 3 2 3 2 4 3 3 3 2 4 3 3 3
##  [5977] 3 4 2 2 3 3 4 3 2 3 3 2 3 4 4 3 4 3 3 3 3 3 3 4 4 3 2 3 2 4 3 3 3 4 2 4
##  [6013] 3 3 3 3 3 4 4 1 1 3 3 2 3 3 3 3 3 4 3 3 3 3 3 3 3 3 3 2 3 1 3 3 3 3 3 2
##  [6049] 3 4 4 3 3 3 3 3 3 3 3 3 2 3 3 3 2 3 3 2 3 2 4 3 4 2 3 3 2 3 4 3 4 2 3 3
##  [6085] 4 4 2 4 3 3 4 3 3 2 4 4 4 3 3 2 4 4 3 3 3 3 3 3 3 3 3 4 3 3 3 4 3 3 4 3
##  [6121] 3 4 3 3 4 3 1 4 3 4 3 4 3 4 3 3 3 3 3 4 1 2 3 3 3 2 3 3 3 3 3 4 2 3 4 3
##  [6157] 3 4 3 4 2 3 4 3 4 3 3 3 2 3 2 2 3 3 3 3 4 3 3 3 3 3 4 4 3 4 4 4 2 3 3 3
##  [6193] 3 4 3 4 4 2 4 3 3 3 2 3 3 2 3 3 3 2 3 2 4 3 3 2 3 3 3 4 3 3 2 4 2 3 3 3
##  [6229] 2 3 3 2 3 4 2 3 2 2 4 3 4 3 4 2 2 3 4 3 3 4 2 4 2 3 3 3 3 2 3 3 3 3 3 2
##  [6265] 3 3 3 2 3 4 3 3 3 3 2 3 4 3 3 2 3 3 3 3 3 4 3 3 3 3 3 3 3 4 3 3 3 3 3 4
##  [6301] 4 2 3 3 3 3 3 2 2 3 3 3 3 3 4 3 3 4 4 3 3 3 3 3 1 4 4 3 3 3 4 3 3 3 3 3
##  [6337] 3 3 4 3 3 3 3 3 2 3 2 3 2 3 3 3 3 3 3 2 2 3 3 3 3 3 3 3 2 3 3 3 4 3 3 4
##  [6373] 3 3 3 4 3 4 3 3 3 3 3 3 4 3 4 3 3 3 3 3 3 4 3 4 3 3 3 3 3 4 3 4 4 3 3 3
##  [6409] 4 3 3 3 3 4 3 3 3 3 3 3 3 3 2 3 2 4 3 4 3 3 3 3 3 2 3 3 3 3 3 4 3 2 4 2
##  [6445] 3 3 3 3 3 4 4 3 3 2 3 3 3 3 3 3 2 3 3 3 3 3 3 4 3 4 3 3 3 3 3 3 3 4 4 4
##  [6481] 3 3 3 2 2 3 4 3 3 4 4 3 3 3 3 2 3 3 4 4 3 4 2 3 3 2 2 3 3 4 3 3 3 3 2 3
##  [6517] 4 2 3 3 3 3 3 3 3 3 4 3 4 4 3 3 4 4 3 3 3 3 4 2 3 4 4 3 2 3 2 3 4 4 3 3
##  [6553] 3 3 4 3 3 2 3 3 2 3 3 4 3 3 3 4 3 3 3 4 3 4 3 3 3 4 3 3 3 3 2 3 3 3 3 4
##  [6589] 4 3 3 3 4 3 3 3 3 3 4 3 3 3 3 3 3 3 3 3 3 3 4 3 3 3 3 3 3 3 2 3 3 2 4 2
##  [6625] 3 3 3 3 4 4 3 3 4 2 3 4 3 3 3 3 3 2 2 3 3 3 3 2 3 3 2 3 3 3 2 3 2 4 3 3
##  [6661] 3 3 3 3 3 4 3 3 3 3 3 3 3 3 2 3 2 2 3 3 3 3 3 3 4 2 4 3 2 3 3 2 3 2 3 3
##  [6697] 3 3 4 3 3 3 2 4 2 3 3 4 4 3 3 3 2 4 4 2 4 2 3 3 3 3 3 4 2 3 3 3 4 3 3 3
##  [6733] 4 3 4 3 3 3 2 3 4 3 3 3 3 4 3 2 2 3 3 3 2 3 3 2 3 3 3 3 3 3 3 3 3 3 3 2
##  [6769] 3 2 4 4 2 2 3 2 3 3 4 3 3 3 3 3 3 2 3 3 4 3 3 3 2 3 3 2 4 3 3 2 3 2 3 3
##  [6805] 3 3 4 3 3 3 4 3 3 4 4 3 3 3 3 3 3 3 4 2 3 3 4 3 3 3 3 3 3 4 3 3 3 3 3 3
##  [6841] 4 3 4 3 3 3 3 3 3 3 4 3 2 3 4 3 3 2 4 2 2 4 3 3 4 2 3 2 2 3 2 3 3 3 4 3
##  [6877] 3 3 3 4 3 3 3 3 3 4 3 3 2 4 2 4 2 2 2 4 4 3 4 2 3 3 3 4 3 4 3 3 3 2 3 3
##  [6913] 3 3 3 3 3 4 4 3 4 4 3 3 3 3 2 3 3 2 4 4 3 4 2 4 4 4 3 3 3 2 3 4 2 4 2 3
##  [6949] 2 3 3 3 4 4 4 3 4 2 2 2 2 3 3 4 2 4 3 3 4 3 3 3 3 4 3 2 3 3 2 4 2 2 4 4
##  [6985] 3 3 1 3 2 3 3 4 3 2 3 2 4 3 2 3 2 3 3 3 4 4 3 3 4 3 3 2 3 3 3 4 3 3 4 4
##  [7021] 3 3 3 3 3 3 3 3 3 4 3 3 3 3 4 3 2 3 3 3 4 3 3 2 3 3 2 3 3 3 3 3 3 3 3 4
##  [7057] 2 2 3 2 3 2 3 3 3 3 4 2 3 4 4 2 2 2 3 3 3 3 4 4 3 2 3 3 3 3 4 4 3 2 3 2
##  [7093] 2 3 2 2 2 2 2 3 2 3 2 3 3 4 3 2 3 4 4 2 3 3 4 3 3 3 3 3 3 3 3 4 3 2 2 2
##  [7129] 2 1 3 2 3 3 3 2 2 4 3 3 3 4 3 3 3 3 4 2 3 3 2 3 4 4 3 2 4 3 3 4 4 3 3 3
##  [7165] 3 3 3 3 3 4 3 4 3 2 3 4 3 3 3 4 3 4 3 3 3 3 3 3 4 3 3 3 3 4 4 3 2 3 4 2
##  [7201] 4 2 3 3 2 3 4 3 4 3 3 4 3 3 3 4 3 3 3 3 4 1 3 3 3 3 3 2 2 3 2 4 3 2 3 3
##  [7237] 3 3 3 3 3 3 3 3 3 2 3 3 4 2 3 4 3 4 2 2 4 3 3 3 3 2 2 2 3 3 3 3 3 4 4 1
##  [7273] 3 4 3 2 3 3 2 3 3 4 3 3 3 3 3 3 4 3 3 2 3 4 4 3 4 3 4 3 2 3 2 3 3 4 4 2
##  [7309] 3 2 3 3 3 3 3 4 3 2 4 2 2 3 3 4 4 2 2 3 3 3 4 3 3 4 4 3 3 3 4 4 3 3 3 4
##  [7345] 4 4 3 3 3 2 3 4 3 3 3 4 2 2 3 3 3 3 3 3 3 3 4 4 3 4 3 3 3 4 3 4 3 3 3 3
##  [7381] 3 4 4 3 3 2 4 2 4 4 3 2 3 3 4 3 3 3 3 4 4 3 3 3 3 3 3 3 3 3 2 4 3 3 3 3
##  [7417] 3 2 1 3 3 2 3 3 3 2 3 3 3 3 3 3 3 3 2 3 2 4 3 3 3 3 3 4 3 3 4 4 4 4 3 3
##  [7453] 3 4 3 2 3 3 3 3 2 3 4 3 3 2 3 3 3 3 3 2 3 3 3 3 3 4 3 3 3 3 2 2 3 3 2 2
##  [7489] 3 3 2 2 3 3 2 4 3 2 3 2 3 2 3 3 4 4 3 4 4 3 3 2 3 4 2 4 2 3 3 2 3 2 4 3
##  [7525] 3 3 4 3 4 2 2 3 2 2 3 4 4 3 4 4 3 3 2 3 3 3 2 4 3 3 3 3 2 3 3 3 2 3 3 3
##  [7561] 4 4 2 4 4 3 3 3 4 4 3 3 2 3 3 3 3 4 2 4 4 3 3 4 3 3 3 3 3 3 3 3 4 3 3 3
##  [7597] 3 3 4 3 3 2 3 3 3 3 3 3 2 4 4 4 4 3 4 3 3 4 3 4 4 3 3 3 2 3 4 3 3 2 3 3
##  [7633] 3 3 3 3 3 3 2 3 4 3 3 3 3 3 2 3 3 3 2 3 2 3 2 4 4 2 3 3 3 3 4 2 3 3 3 3
##  [7669] 3 4 2 3 2 4 3 3 2 4 3 4 3 3 3 3 3 3 3 4 4 3 4 3 3 3 3 3 3 3 3 3 3 2 3 3
##  [7705] 2 3 3 3 3 2 3 3 3 3 3 3 3 4 3 3 3 4 3 4 4 3 4 3 3 3 3 3 3 3 3 3 3 3 3 3
##  [7741] 3 3 3 3 4 3 3 3 3 4 4 3 3 2 3 4 4 3 3 2 3 4 3 2 3 3 2 3 3 4 4 2 3 3 3 3
##  [7777] 3 2 2 3 3 2 2 3 3 2 3 2 2 3 2 4 3 3 3 4 3 3 3 3 4 3 3 3 3 3 3 3 3 4 2 1
##  [7813] 3 3 3 4 3 3 3 3 2 3 3 2 3 3 3 3 3 3 3 3 3 3 3 4 3 3 4 2 3 3 3 2 3 2 3 3
##  [7849] 3 3 2 3 4 3 4 3 3 2 3 2 3 3 3 4 4 3 3 3 2 4 3 3 3 3 4 3 3 3 3 4 4 2 3 3
##  [7885] 3 3 4 3 3 3 3 3 3 3 4 3 3 3 3 3 3 4 3 3 3 3 3 3 3 4 3 3 3 3 3 3 3 4 2 3
##  [7921] 3 4 3 3 4 3 3 3 3 4 4 3 3 3 3 3 3 3 3 3 3 3 2 3 3 3 3 3 3 2 3 3 3 4 2 3
##  [7957] 3 4 3 3 3 4 3 4 3 4 3 3 4 3 3 3 3 3 3 4 2 3 3 3 3 3 3 4 3 3 3 3 3 2 3 3
##  [7993] 3 4 3 2 3 4 3 3 3 4 3 3 3 3 3 4 3 3 3 3 3 4 4 3 2 2 4 3 4 2 4 3 2 2 4 4
##  [8029] 3 3 4 2 3 3 3 2 3 3 2 3 4 4 3 2 2 4 4 3 2 4 4 4 3 3 2 3 3 4 4 4 4 3 3 3
##  [8065] 3 3 4 3 4 3 3 3 3 3 3 4 4 2 3 3 4 3 3 3 2 3 3 3 3 3 3 3 4 3 4 3 3 3 2 3
##  [8101] 2 2 3 4 4 3 3 3 3 3 4 3 4 3 4 2 3 3 3 2 3 4 4 3 2 3 4 2 3 2 4 4 3 3 3 3
##  [8137] 3 4 2 3 3 3 2 4 4 4 4 3 3 3 3 3 3 2 3 3 3 2 2 4 2 3 2 3 4 4 3 3 4 3 3 3
##  [8173] 3 2 3 2 3 3 3 3 4 4 3 3 3 4 3 4 4 3 3 3 4 3 3 4 4 3 3 4 2 4 3 3 3 2 3 3
##  [8209] 3 3 3 3 1 3 4 3 3 3 2 4 3 2 2 3 3 3 3 3 3 3 2 4 2 3 3 3 3 4 4 3 4 2 3 4
##  [8245] 3 4 3 3 3 3 3 4 3 4 2 3 3 3 1 3 3 2 3 4 3 4 3 3 3 3 2 2 3 3 3 2 2 4 3 4
##  [8281] 3 3 3 3 3 3 2 3 3 2 2 3 3 4 4 3 3 2 3 4 3 4 2 3 3 3 3 3 4 4 3 4 3 3 3 3
##  [8317] 2 3 3 3 3 3 3 4 3 3 3 3 3 3 3 3 3 3 3 4 2 4 2 3 3 4 3 3 3 3 3 3 3 3 4 3
##  [8353] 3 3 2 4 4 3 3 4 3 4 2 4 3 4 1 3 3 2 3 3 3 4 3 3 4 3 3 2 4 2 3 4 3 2 3 3
##  [8389] 3 3 4 3 3 3 3 4 3 3 3 3 2 3 3 3 3 4 3 3 3 3 2 4 1 3 3 4 3 3 3 3 3 1 4 3
##  [8425] 3 3 3 4 3 3 3 3 2 3 3 1 3 4 4 3 3 3 3 3 3 4 2 1 3 3 3 3 3 3 3 3 3 3 2 2
##  [8461] 3 3 3 3 4 3 3 3 4 1 3 3 4 3 3 2 4 2 4 4 3 3 3 3 3 3 2 3 2 3 3 3 4 3 3 2
##  [8497] 2 3 3 4 4 3 3 2 3 3 3 4 4 3 3 3 3 4 4 4 3 3 4 3 3 3 3 3 3 2 3 2 4 3 2 3
##  [8533] 3 3 3 3 2 3 3 3 2 4 4 3 3 3 3 4 3 4 2 3 3 3 3 2 4 2 3 3 3 3 3 3 4 3 3 3
##  [8569] 4 3 3 3 3 3 3 3 4 3 4 3 4 3 3 3 3 3 3 3 3 3 3 4 3 4 2 3 3 3 4 3 2 3 3 2
##  [8605] 3 3 3 3 2 3 3 3 3 4 3 3 3 3 4 2 3 4 3 3 4 2 3 4 3 3 3 3 3 2 3 3 3 3 2 3
##  [8641] 2 4 2 3 3 3 2 3 2 3 3 4 3 4 3 3 4 3 4 3 2 4 3 3 3 3 4 2 4 4 3 3 2 2 4 3
##  [8677] 3 4 3 3 3 2 3 4 2 2 2 4 4 3 2 3 4 2 3 3 2 3 3 2 4 4 3 3 3 4 4 3 3 3 3 3
##  [8713] 4 4 3 3 3 3 4 4 4 3 2 3 3 3 3 2 4 3 4 3 4 4 4 3 3 3 3 3 3 3 3 3 3 3 3 2
##  [8749] 3 3 3 2 3 2 4 2 4 2 3 2 2 4 3 3 4 3 3 2 2 3 4 3 4 3 3 2 2 2 4 2 4 3 3 3
##  [8785] 4 2 3 2 4 4 4 2 4 4 2 3 2 3 3 3 3 4 3 3 3 4 2 4 2 2 3 2 3 3 3 3 2 3 4 4
##  [8821] 3 2 3 3 4 4 3 3 3 2 4 4 3 2 3 4 3 2 3 2 3 3 3 3 2 3 2 2 2 2 3 2 3 4 3 3
##  [8857] 2 3 4 2 2 3 3 2 3 3 3 2 3 3 3 3 4 3 2 2 3 3 3 2 3 3 3 2 3 2 3 3 3 2 2 4
##  [8893] 3 3 4 3 3 2 3 2 3 4 4 3 3 3 3 2 3 3 3 3 3 2 3 3 3 3 3 3 4 3 4 4 3 3 3 4
##  [8929] 3 1 2 3 2 3 3 3 3 4 3 3 3 3 2 2 3 2 4 2 2 4 3 3 3 2 4 3 3 3 4 3 3 2 3 3
##  [8965] 3 3 4 3 3 3 3 4 2 3 3 3 2 2 3 3 3 3 2 2 3 2 3 3 4 3 4 3 3 2 3 3 3 4 2 3
##  [9001] 2 4 2 3 3 4 4 3 4 4 4 3 3 3 2 3 2 3 3 4 2 4 4 4 4 3 4 4 3 2 4 2 3 3 4 2
##  [9037] 3 2 3 3 4 3 3 2 3 4 3 3 2 3 3 4 4 4 2 4 3 4 3 2 2 3 2 4 3 4 2 3 3 3 2 3
##  [9073] 3 4 3 3 4 4 3 3 3 3 3 3 3 3 3 3 3 4 3 4 3 4 4 3 3 3 3 3 3 3 4 3 3 3 3 3
##  [9109] 3 4 3 3 3 4 3 3 3 3 4 3 3 3 3 3 2 4 3 3 4 3 3 4 3 4 3 2 3 4 3 3 2 2 4 3
##  [9145] 3 3 4 3 4 3 4 3 3 3 3 3 3 4 4 3 3 4 3 3 4 3 4 2 4 3 3 3 4 3 4 3 3 3 3 2
##  [9181] 3 3 3 4 3 4 3 3 3 3 4 3 3 2 3 3 4 3 3 4 4 3 2 2 3 3 3 3 3 3 3 3 3 3 4 3
##  [9217] 4 3 3 4 2 3 3 3 3 3 3 3 4 3 3 4 3 3 3 3 4 4 3 4 3 3 3 3 3 3 3 3 2 2 3 4
##  [9253] 4 3 2 3 3 3 3 3 4 3 3 3 3 3 3 3 3 3 3 3 4 3 3 4 4 3 3 3 3 3 3 3 3 3 2 3
##  [9289] 3 3 3 3 3 3 3 3 3 3 3 4 3 4 3 3 3 3 3 4 3 3 3 3 3 3 4 3 2 4 4 3 3 3 2 3
##  [9325] 3 2 4 3 4 3 4 3 3 3 3 4 3 3 3 3 3 2 3 3 4 3 3 3 4 2 4 3 3 3 3 3 3 2 3 4
##  [9361] 3 3 3 3 3 4 3 3 3 4 4 4 3 3 2 4 4 3 3 2 2 2 3 3 3 3 3 3 2 4 3 3 3 2 3 4
##  [9397] 3 4 2 3 3 3 3 3 3 2 4 4 4 4 4 3 4 4 4 4 3 3 3 2 2 3 4 4 3 2 3 3 4 3 3 3
##  [9433] 4 4 4 3 3 4 4 3 3 3 4 4 3 2 3 3 2 2 2 3 4 3 2 4 2 3 3 3 4 2 3 3 3 4 4 4
##  [9469] 4 3 3 2 3 3 4 3 3 3 3 4 3 3 4 4 4 3 3 3 3 3 2 2 3 2 3 3 4 3 3 3 3 3 3 4
##  [9505] 1 4 3 3 3 3 3 2 3 4 3 3 2 3 2 3 2 3 4 3 3 3 4 4 3 4 3 3 3 3 4 2 3 4 3 3
##  [9541] 3 4 3 3 3 2 3 3 3 2 4 1 3 3 3 3 3 3 3 3 3 3 2 3 2 2 3 4 3 3 3 3 3 2 3 4
##  [9577] 4 2 4 3 4 4 4 4 3 3 3 4 3 3 3 3 3 3 4 3 3 3 4 3 4 3 3 3 3 3 2 4 4 3 4 2
##  [9613] 2 3 4 4 4 3 4 4 3 1 3 3 3 3 1 4 3 3 3 2 3 2 4 3 2 3 3 3 4 3 4 3 3 2 4 4
##  [9649] 2 2 3 4 3 3 3 3 3 4 2 3 4 4 3 4 4 3 3 3 3 4 4 3 4 3 4 3 3 3 3 3 3 3 3 2
##  [9685] 1 3 3 4 4 3 2 3 3 2 3 3 3 2 4 3 2 3 4 3 3 3 2 3 4 3 3 4 3 3 3 3 3 3 3 3
##  [9721] 3 3 3 3 3 4 4 3 3 3 3 3 4 2 3 3 4 3 3 3 4 3 4 3 3 2 3 4 4 4 4 2 4 4 2 2
##  [9757] 3 3 4 3 4 2 4 3 3 4 4 3 4 3 4 4 3 4 4 3 3 3 3 4 4 4 3 4 3 3 4 2 4 3 3 2
##  [9793] 3 3 4 4 3 2 3 3 3 3 3 4 4 3 3 3 3 3 4 4 2 2 3 3 3 3 4 3 4 4 4 3 4 4 3 3
##  [9829] 3 4 3 3 3 3 4 4 4 3 4 3 3 2 3 3 3 3 2 2 3 3 3 4 4 3 3 3 3 3 2 3 2 4 3 3
##  [9865] 3 3 4 3 2 3 3 2 3 3 3 3 2 3 4 3 3 4 3 2 3 2 3 4 3 4 3 4 4 3 3 3 4 3 4 4
##  [9901] 3 4 3 3 2 4 3 3 4 4 3 2 3 2 3 4 2 3 3 4 3 4 3 3 3 3 3 2 4 4 3 2 4 2 3 3
##  [9937] 3 3 3 3 3 3 4 3 3 3 3 4 4 3 4 4 3 3 4 4 3 3 3 4 3 3 3 2 4 4 3 3 4 4 3 4
##  [9973] 4 2 3 3 3 4 4 3 4 3 4 3 3 4 2 4 4 3 3 3 3 4 4 4 4 3 3 3 4 4 3 4 4 1 3 3
## [10009] 3 4 4 3 4 4 3 3 2 4 3 3 3 4 4 3 3 3 3 3 4 2 3 3 3 3 4 4 3 2 3 3 3 3 3 2
## [10045] 3 3 3 2 4 4 3 2 3 2 3 4 2 3 3 3 4 3 3 4 4 3 3 3 4 3 2 2 4 3 3 2 4 3 3 4
## [10081] 4 3 4 3 3 4 4 3 2 2 3 3 3 3 3 3 4 3 3 2 4 4 4 2 4 3 4 2 3 3 3 3 3 3 3 3
## [10117] 3 4 3 3 4 2 4 2 3 3 3 3 2 3 3 1 3 2 4 3 4 4 3 3 3 3 4 1 4 2 3 2 2 4 4 4
## [10153] 4 3 3 4 4 4 3 3 3 3 3 3 4 4 2 3 2 3 3 2 4 4 3 2 3 4 4 4 3 3 3 4 3 2 4 4
## [10189] 3 4 4 3 4 3 2 3 3 3 3 2 3 3 4 4 4 3 2 3 4 4 3 3 4 3 4 2 3 4 3 4 4 3 4 2
## [10225] 3 4 4 3 3 3 3 4 2 4 4 4 2 2 4 3 4 3 2 4 4 3 3 4 2 4 4 3 3 4 3 3 3 4 4 4
## [10261] 4 4 4 3 4 4 4 4 3 2 3 4 2 3 3 3 3 4 4 3 3 4 4 4 3 3 3 2 2 3 2 3 4 4 3 3
## [10297] 3 3 3 4 3 3 3 4 4 4 4 3 4 3 4 3 4 3 3 3 4 2 3 3 3 3 3 4 3 3 3 4 4 2 3 2
## [10333] 2 3 4 3 4 4 2 4 3 2 4 3 4 3 4 3 2 2 3 4 2 3 2 4 3 2 2 2 4 4 3 3 3 4 3 4
## [10369] 2 4 4 4 2 4 2 3 4 3 2 4 2 3 3 3 3 3 3 3 4 3 4 4 4 4 4 3 3 3 2 3 3 4 3 4
## [10405] 3 2 4 4 3 4 2 3 3 4 4 3 4 2 4 4 3 4 4 4 3 4 4 3 3 3 4 4 2 3 4 2 2 4 2 3
## [10441] 4 2 4 3 4 3 2 4 2 2 2 4 3 2 2 4 4 4 4 3 3 4 3 4 3 3 3 4 3 3 4 3 2 4 2 2
## [10477] 4 3 4 2 3 4 3 3 2 2 4 4 3 4 3 2 3 4 3 4 3 4 1 2 4 3 3 4 3 3 2 4 2 3 3 4
## [10513] 1 4 4 3 3 3 3 4 3 3 3 4 4 4 3 4 3 3 4 3 4 3 4 4 3 2 2 3 2 2 4 4 2 3 4 3
## [10549] 4 3 4 4 3 3 3 4 3 4 4 4 4 3 3 4 4 4 3 3 3 4 3 4 3 3 3 3 4 3 4 3 3 4 3 4
## [10585] 3 4 3 3 4 3 4 4 3 4 3 3 3 3 3 3 3 3 4 4 3 4 3 2 4 4 4 3 3 3 3 4 4 4 4 3
## [10621] 3 3 3 4 3 3 3 3 2 3 4 4 3 3 3 3 3 4 4 3 3 3 2 3 3 3 3 3 3 3 3 3 3 3 3 3
## [10657] 3 4 3 4 4 4 3 4 4 3 4 4 3 3 3 3 3 3 4 2 4 3 3 3 3 3 4 4 4 4 2 3 3 3 2 3
## [10693] 4 3 3 4 4 4 4 3 4 3 4 3 3 3 3 2 1 4 3 3 3 3 3 3 4 3 3 3 2 3 4 4 3 3 4 4
## [10729] 4 4 4 4 1 3 3 3 2 4 4 4 4 3 4 3 3 3 3 2 1 4 3 3 4 3 3 3 3 4 3 3 3 3 3 3
## [10765] 3 3 3 3 4 3 4 3 2 3 3 4 2 2 2 4 2 2 4 2 3 4 2 3 3 3 4 3 2 4 3 4 3 3 3 4
## [10801] 4 3 3 3 4 3 4 3 3 4 3 3 3 4 4 3 4 3 2 4 4 3 4 4 3 4 3 3 3 4 4 2 4 4 2 2
## [10837] 4 4 3 4 3 4 3 1 3 3 3 3 3 3 4 3 2 3 4 3 4 3 4 3 4 4 4 4 4 3 3 4 4 3 3 2
## [10873] 3 3 3 3 4 3 3 4 3 4 4 3 4 4 3 4 2 4 3 3 3 3 4 4 3 3 3 4 3 2 1 4 4 3 4 4
## [10909] 3 4 4 4 3 4 3 4 3 3 3 3 4 3 3 3 3 4 4 3 3 4 4 4 4 4 3 3 2 3 4 2 3 3 3 4
## [10945] 3 4 3 4 3 3 4 3 4 3 4 3 3 3 2 4 3 4 4 4 4 4 4 2 4 3 3 4 4 2 3 4 4 2 4 4
## [10981] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 2 4 4 4 4 4
## [11017] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 2 2 4 4 4
## [11053] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 2 2 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [11089] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 2 4 4 4 4 4 4 4 4 4 4
## [11125] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 2 4 2 4 4 4 4 4 4 4 4 4
## [11161] 4 2 4 4 4 4 4 4 2 4 2 4 4 4 4 4 4 4 4 4 4 4 4 2 4 2 4 4 4 4 4 4 4 4 4 4
## [11197] 4 1 2 4 4 4 4 4 4 4 4 4 2 4 4 4 4 4 4 2 4 4 4 2 4 4 4 4 4 4 4 4 4 4 4 4
## [11233] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [11269] 4 4 2 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [11305] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 2 4 2 4 4 4 4 4 4 4 4 4 2 4 4
## [11341] 4 4 4 2 4 4 4 4 4 4 4 4 4 2 4 4 4 4 4 2 4 4 4 4 4 4 4 2 4 4 4 4 4 4 4 4
## [11377] 4 4 4 4 4 3 4 1 3 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 2 4 4 4 4 4 4 4 4 4
## [11413] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 2 4 4
## [11449] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 2 4 4
## [11485] 4 4 4 2 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 1
## [11521] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 2 4 4 4 4 2 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [11557] 4 2 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [11593] 4 4 4 4 4 4 2 4 4 4 4 4 4 4 4 4 2 4 4 4 4 4 4 4 4 4 4 4 4 4 2 4 4 4 4 4
## [11629] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 2 4 2 4 2 4 4 4
## [11665] 2 4 2 4 4 4 4 4 4 4 4 4 4 2 4 4 4 4 4 2 4 4 4 4 4 4 4 4 4 4 4 4 1 4 4 2
## [11701] 4 2 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 2 4 4 4 4 4 4 4 4 4 4 4 4
## [11737] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [11773] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [11809] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 2 4
## [11845] 4 4 4 2 4 4 4 2 4 4 2 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [11881] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 2 4 4 4 4 4
## [11917] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 1 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [11953] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [11989] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [12025] 4 1 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [12061] 4 4 4 4 2 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 2 4 2 4
## [12097] 4 4 4 4 4 4 4 2 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [12133] 4 4 4 4 4 4 2 4 4 2 4 4 4 4 4 4 4 2 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 2
## [12169] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 2 4 4 4
## [12205] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 1 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 2
## [12241] 4 4 4 4 4 4 4 4 4 4 4 4 2 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [12277] 4 4 4 4 4 4 4 2 4 4 4 4 4 4 4 4 4 4 4 2 4 4 4 2 4 2 4 4 4 4 4 4 4 4 4 4
## [12313] 4 4 4 4 4 2 4 4 4 1 4 2 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [12349] 2 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 2 4 4 4 4 4 4 4 4 4 4 4 4 4
## [12385] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 2 4 4 4 4 4 4 4 4 4
## [12421] 4 4 4 4 4 4 4 4 4 2 4 4 2 4 4 4 2 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 2 4 4
## [12457] 2 4 4 4 4 4 4 4 4 4 4 4 4 4 2 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [12493] 4 4 4 4 4 4 4 4 4 4 4 2 4 4 4 4 2 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [12529] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [12565] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [12601] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [12637] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 2
## [12673] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 1 4 4 4 4 4 4 4 1 4 4 1 4 4 4 4 4 4 4
## [12709] 1 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 2 4 4 2 4 4 4 4 4 4
## [12745] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [12781] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [12817] 2 4 4 4 4 4 4 4 4 4 4 4 4 4 2 4 4 4 4 4 4 4 4 2 4 2 4 4 4 4 4 4 4 4 4 4
## [12853] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 2 2 4 4 4 4 4 4 4 4 4 4 4
## [12889] 4 4 4 4 4 4 2 4 4 4 4 4 4 4 4 4 4 4 4 4 4 2 4 4 4 4 4 4 4 4 4 4 4 2 4 4
## [12925] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [12961] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [12997] 4 4 4 4 4 4 2 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 2 4 4 4 4 4 4 4 4 4 4 4 4
## [13033] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 2
## [13069] 4 4 4 4 4 4 4 4 4 2 4 4 4 4 2 4 4 4 4 4 4 4 4 4 4 2 4 4 4 4 4 4 4 4 4 4
## [13105] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 2 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [13141] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 1 4 4 2 4 4 2 4 4 4 4 4 4 4 4 4 4 4 4 4 1 4
## [13177] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [13213] 2 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [13249] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 2 4 4 4 4 4 4 2 4 4 4 4 4 4 4 4 4
## [13285] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [13321] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 2 4 4 4 4
## [13357] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 1 4 4
## [13393] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 2 4 4 4 4 4 4 4
## [13429] 4 4 4 4 4 4 4 4 4 4 4 1 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [13465] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 2 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 2 2 4 4 4
## [13501] 4 4 4 4 4 1 4 1 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [13537] 4 4 4 4 2 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [13573] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 2 4 4 4 4 4 4 4 4 4 4
## [13609] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 2
## [13645] 4 4 4 4 4 4 4 4 4 4 4 4 4 1 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [13681] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [13717] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 1 4
## [13753] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 2 4 4 4 4 4 4
## [13789] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [13825] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 2 4 4 4 4 4 4 4 4 4
## [13861] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [13897] 2 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 2 4 4 4 4 4 4 4
## [13933] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [13969] 4 4 4 4 4 4 4 4 4 1 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [14005] 4 4 4 2 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [14041] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 2 4 4 4 4 4
## [14077] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [14113] 4 4 4 4 4 2 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 2 4 4 4 4 4 4 4 4 4 4 4 2 4 4
## [14149] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 2 4 4 4 4 4 4 4 4 4 4
## [14185] 1 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 2 4 4 4 4 4 4 4 4 4 4 4 2 4 4 4
## [14221] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [14257] 4 4 1 4 4 4 4 4 4 4 4 4 4 4 2 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [14293] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 2 4
## [14329] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [14365] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [14401] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [14437] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 2 4 4
## [14473] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 1 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [14509] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 2 4 4 4 4 4 4 4 4 4 4 4 4 4
## [14545] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 2 4 4 4 4 4 4 4 4
## [14581] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 2 4 4
## [14617] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [14653] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 2 4 4 4 4 4 4 4 4 4 4 4
## [14689] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 2 4 4 4 4 4 4 4 4 4 4 4 4 4
## [14725] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [14761] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [14797] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 2 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [14833] 4 4 4 4 4 4 4 4 4 4 4 2 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [14869] 4 4 4 4 4 4 4 4 4 4 4 4 2 4 4 4 4 4 4 4 4 4 4 2 4 4 4 4 4 4 4 4 4 4 4 4
## [14905] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [14941] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [14977] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [15013] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [15049] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [15085] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [15121] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [15157] 4 4 4 4 4 4 2 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [15193] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [15229] 4 4 4 4 4 4 4 4 2 4 4 4 4 4 4 4 2 4 1 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [15265] 4 4 4 4 4 4 4 4 4 4 4 4 2 4 4 4 1 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [15301] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 2 4 4 4 4 4 1 4 4 1 4 4 4 4 4 4 4 4 4 4
## [15337] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [15373] 4 4 4 1 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [15409] 4 4 4 4 4 4 4 4 4 4 4 1 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [15445] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [15481] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 1 1 4 4
## [15517] 4 4 4 4 4 4 4 1 4 4 4 4 4 4 4 4 4 4 4 4 4 1 4 4 4 4 4 4 4 4 4 4 2 4 4 4
## [15553] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [15589] 4 4 4 1 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [15625] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 2 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [15661] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [15697] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [15733] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [15769] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [15805] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [15841] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [15877] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [15913] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [15949] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 1 4 4 4
## [15985] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [16021] 4 4 4 4 4 4 4 4 1 4 4 4 4 4 4 4 1 4 4 4 4 4 4 4 1 4 4 4 4 4 4 4 4 4 4 4
## [16057] 4 4 1 4 4 4 4 4 4 4 4 4 4 4 4 4 2 4 4 4 4 4 4 4 4 1 4 4 4 4 4 4 4 4 4 4
## [16093] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 1 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [16129] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [16165] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [16201] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [16237] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 2
## [16273] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [16309] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [16345] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [16381] 4 4 4 4 4 4 2 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [16417] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [16453] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [16489] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [16525] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [16561] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [16597] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [16633] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [16669] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [16705] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [16741] 2 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [16777] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [16813] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [16849] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [16885] 4 4 4 4 4 4 4 4 4 4 1 4 4 4 4 4 4 4 4 4 1 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [16921] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [16957] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [16993] 4 4 2 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [17029] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 1 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [17065] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 1 4 4 4 4 1 4 4 4 4 4 4 4 4 1 4 4 1 4 4 4 4
## [17101] 4 4 4 4 4 4 4 4 1 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 1
## [17137] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 1 4 4 4 4 4 4
## [17173] 4 4 4 4 4 4 1 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [17209] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [17245] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [17281] 4 1 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [17317] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 1 4 1 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [17353] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [17389] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [17425] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [17461] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [17497] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [17533] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [17569] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [17605] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [17641] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [17677] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [17713] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [17749] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [17785] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [17821] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [17857] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [17893] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [17929] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [17965] 4 4 4 4 4 4 4 1 4 4 4 4 4 4 4 4 1 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [18001] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [18037] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [18073] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 1 4 4 4 4
## [18109] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [18145] 4 4 4 4 4 4 4 4 4 1 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [18181] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [18217] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [18253] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 1 4 4 4 4 4
## [18289] 4 4 4 1 4 1 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [18325] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [18361] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## [18397] 4 4 4 4 4 4 1 4 4 4 4 4 4 4 4 4 4 4 4 4 4
## 
## Within cluster sum of squares by cluster:
## [1] 3905 4397 2835 5105
##  (between_SS / total_SS =  70.6 %)
## 
## Available components:
## 
## [1] "cluster"      "centers"      "totss"        "withinss"     "tot.withinss"
## [6] "betweenss"    "size"         "iter"         "ifault"

# Append the clustering result to A
A$grp = k4$cluster

# TODO: Show how many customers are there in each group
table(A$grp)

## 
##    1    2    3    4 
##  127 2466 6322 9502

Discussion

Try applying K-means to the dataset with different numbers of clusters:

What do you think is the optimal number of clusters?
Do you gain more insights with the new clustering result?

Visualizing The Clusters

# Group A by column "grp"
group_by(A, grp) %>%
  # Computes summary statistics for each group  
  summarise(
  recent=mean(recent), 
  freq=mean(freq), 
  money=mean(money), 
  size=n() ) %>% 
  # Add a new column "revenue" to the data: total revenue contribution of each group in thousands
  mutate( revenue = size*money/1000 )  %>% 
  # Exclude groups that consist of only one observation
  filter(size > 1) %>% 
  # Initializes a ggplot graph with "freq" and "money" as the x and y axes.
  ggplot(aes(x=freq, y=money)) +
  # Add points to the plot, with point size mapped to "revenue" and color mapped to "recent"
  geom_point(aes(size=revenue, col=recent),alpha=0.5) +
  # Adjust the scale of point sizes to range from 4 to 40
  scale_size(range=c(4,40)) +
  # Create a color gradient from green (low values of "recent") to red (high values)
  scale_color_gradient(low="green",high="red") +
  # Apply a logarithmic transformation to both axes
  scale_x_log10() + scale_y_log10() + 
  # Add text labels to each point, displaying the group size
  geom_text(aes(label = size )) +
  # Set white background and remove the legend
  theme_bw() + guides(size=F) +
  # Set up the plot labels and titles
  labs(title="Customer Segements",
       subtitle="(bubble_size:revenue_contribution; text:group_size)",
       color="Recency") +
      xlab("Frequency (log)") + ylab("Average Transaction Amount (log)")

## Warning: The `<scale>` argument of `guides()` cannot be `FALSE`. Use "none" instead as
## of ggplot2 3.3.4.
## This warning is displayed once every 8 hours.
## Call `lifecycle::last_lifecycle_warnings()` to see where this warning was
## generated.

Discussion

Observe the diagram above:

Do you see a clear segmentation of “new customers”, “high-value customers”, and ” frequent customers”?
In this dataset, which group of customers do you think is the most valuable?

Rule-based Segmentation

Customer Segmentation Rules

STS = c("N1","N2","R1","R2","S1","S2","S3")

# Write a function to assign group label to each customer according to the rule above
# K is the average purchase cycle of returning customers
Status = function(rx,fx,mx,sx,K) {factor(
  ifelse(sx < 2*K,
         ifelse(fx*mx > 50, "N2", "N1"),
         ifelse(rx < 2*K,
                ifelse(sx/fx < 0.75*K,"R2","R1"),
                ifelse(rx < 3*K,"S1",
                       ifelse(rx < 4*K,"S2","S3")))), STS)}

Customer Data in A Period

# Create an empty list
Y = list()

# Consolidate customer data into a data frame at the end of each year
for(y in 2010:2015) {
  # End-of-period dates for the current and previous periods
  D = as.Date(paste0(c(y, y-1),"-12-31")) 
  # Starting with transaction data
  Y[[paste0("Y",y)]] = X %>%
    # Cut data to the end-of-period date
    filter(date <= D[1]) %>% 
    # Days since the transaction to the end of the period
    mutate(days = 1 + as.integer(D[1] - date)) %>%   
    group_by(cid) %>% summarise(    # Summarize by customer
      recent = min(days),           #   Days since last purchase to the end of the period
      freq = n(),                   #   Number of purchases (up to the end of the period)
      money = mean(amount),         #   Average purchase amount (up to the end of the period)
      senior = max(days),           #   Days since the first purchase to the end of the period
      status = Status(recent,freq,money,senior,K),  # Status at the end of the period
      since = min(date),                      # Date of first purchase
      y_freq = sum(date > D[2]),              # Number of purchases in the current period
      y_revenue = sum(amount[date > D[2]])    # Revenue from purchases in the current period
    ) %>% 
    # Convert to Data Frame and Store
    data.frame 
  }

# Show the number of rows (i.e., the number of customers processed) for each year's data frame
sapply(Y, nrow)

## Y2010 Y2011 Y2012 Y2013 Y2014 Y2015 
## 10407 11674 13562 15468 16905 18417

# Show the number of customers in each group across the years
sapply(Y, function(x) table(x$status))

##    Y2010 Y2011 Y2012 Y2013 Y2014 Y2015
## N1  3330  2640  2778  3054  3106  2772
## N2  1655  1496  1704  1918  2091  1957
## R1  1298  1731  2177  2156  2102  2140
## R2  1547  1831  1923  1955  2016  2264
## S1  2203  2163  2147  2357  1963  2807
## S2   360  1502  1820  1723  2110  1718
## S3    14   311  1013  2305  3517  4759

Visualize The Changes in Group Size

# Define color palette
cols = c("gold","orange","blue","green",
         "pink","magenta","darkred")

# Create bar plot to show the number of customers in each group across the years
sapply(Y, function(df) table(df$status)) %>% barplot(col=cols)
legend("topleft",rev(STS),fill=rev(cols))

Dynamic Analysis of Features

# Combining Data Across Years
# Apply a function to each element of the list Y, where each d is a data frame representing customer data for a specific year
CustSegments <- do.call(rbind, lapply(Y, function(d) {
  d %>%
    # Group the data within each year's data frame by the status column
    group_by(status) %>%
    # Calculate several summary statistics for each status group
    summarise(
      average_frequency = mean(freq),
      average_amount = mean(money),
      total_revenue = sum(y_revenue),
      total_no_orders = sum(y_freq),
      average_recency = mean(recent),
      average_seniority = mean(senior),
      group_size = n(),
      .groups = 'drop'
    )
})) %>%
ungroup() %>%
# Add a new column, 'year', to the data frame, recording the year on which the summary statistics of each row are based.
mutate(year = rep(2010:2015, each = 7))

CustSegments

# Create a scatter plot to show how the attributes of each customer segment changes over time
p <- ggplot(CustSegments, 
  # Set the x-axis to represent the average frequency of transactions per customer within each segment, the y-axis to represent the average transaction amount per customer within each segment
  # Map the size of the points to the number of customers in each segment (group_size)
  # Color the points according to the status of each customer segment
  aes(x = average_frequency, y = average_amount, size = group_size, color = status)) +
  # Add scatter plot points to the graph
  geom_point() +
  # Create separate panels (facets) for each value of year in the data frame
  facet_wrap(~year) +
  # Set up the plot labels and titles
  labs(title = "Customer Segments Over Time",
       x = "Average Frequency",
       y = "Average Amount")

# Show the plot
ggplotly(p)

Discussion

Observe the scatter plot above:

What does each component in the plot represent? E.g., the size of the points, the color of the points, the x-axis, and the y-axis…
How do the group size and attributes change over time? Why?
Compare this plot with the cluster visualization in K-means clustering analysis. Can we find correspondences between the rule-based groups and the K-means clusters?

Customer Flows between Groups

# Observe the how customers transition from one group in 2011 to another in 2012
# Select "cid" and "status" from data in 2011 and 2012, and merge them based on "cid"
df = merge(Y$Y2011[,c(1,6)], Y$Y2012[,c(1,6)],
           by="cid", all.x=T)

# Create a contingency table of the statuses between the two years
# The table counts how many times each status in 2011 transitions to each status in 2012
tx = table(df$status.x, df$status.y) %>% 
  as.data.frame.matrix() %>% as.matrix()

tx

##      N1  N2   R1   R2  S1   S2  S3
## N1 1424 309  166   32 709    0   0
## N2    0 861  206  326 103    0   0
## R1    0   0 1071   49 611    0   0
## R2    0   0  269 1512  50    0   0
## S1    0   0  247    4 674 1238   0
## S2    0   0  190    0   0  582 730
## S3    0   0   28    0   0    0 283

# TODO: Convert values in tx to be percentage
tx %>% prop.table(1) %>% round(4)

##        N1     N2     R1     R2     S1     S2    S3
## N1 0.5394 0.1170 0.0629 0.0121 0.2686 0.0000 0.000
## N2 0.0000 0.5755 0.1377 0.2179 0.0689 0.0000 0.000
## R1 0.0000 0.0000 0.6187 0.0283 0.3530 0.0000 0.000
## R2 0.0000 0.0000 0.1469 0.8258 0.0273 0.0000 0.000
## S1 0.0000 0.0000 0.1142 0.0018 0.3116 0.5724 0.000
## S2 0.0000 0.0000 0.1265 0.0000 0.0000 0.3875 0.486
## S3 0.0000 0.0000 0.0900 0.0000 0.0000 0.0000 0.910

Visualizing The Customer Flow

# Use the package "chorddiag" to visualize tx
chorddiag(tx, groupColors=cols)

Discussion

Visualize the customer flows from 2014 to 2015 and discuss the following:

Compare the results from 2011-2012 and 2014-2015. How do the customer flows differ?
Have you observed any patterns in the customer flow? Do customers from one group consistently transition from the same source groups?

Learning to Predict

We are now going to train a model using data collected in 2015 to predict

Whether the customer will retain in 2016 (Retain)
How much this customer will spend in 2016 (Revenue)

Data Preprocessing

We first train a model that uses data collected in 2014 to predict Retain and Revenue in 2015.

# TODO: Explain what this piece of code does
CX = left_join(Y$Y2014, Y$Y2015[,c(1,8,9)], by="cid")
names(CX)[8:11] = c("freq0","revenue0","Retain", "Revenue")
CX$Retain = CX$Retain > 0

# Compute the retention rate
table(CX$Retain) %>% prop.table()

## 
##  FALSE   TRUE 
## 0.7701 0.2299

Predicting Retention

Now, we are going to learn a logistic regression classifier for retention prediction.

# Train a logistic regression model, with Y=Retain and X=CX[,c(2:3,6,8:10)]
mRet = glm(Retain ~ ., CX[,c(2:3,6,8:10)], family=binomial())

# Show the model training results
summary(mRet)

## 
## Call:
## glm(formula = Retain ~ ., family = binomial(), data = CX[, c(2:3, 
##     6, 8:10)])
## 
## Coefficients:
##              Estimate Std. Error z value Pr(>|z|)    
## (Intercept) -1.074007   0.089431  -12.01  < 2e-16 ***
## recent      -0.002067   0.000131  -15.73  < 2e-16 ***
## freq         0.095217   0.013882    6.86  6.9e-12 ***
## statusN2     0.669429   0.070234    9.53  < 2e-16 ***
## statusR1     0.488321   0.084389    5.79  7.2e-09 ***
## statusR2     1.290002   0.110841   11.64  < 2e-16 ***
## statusS1     0.670604   0.146532    4.58  4.7e-06 ***
## statusS2     1.353554   0.208210    6.50  8.0e-11 ***
## statusS3     2.573689   0.275786    9.33  < 2e-16 ***
## freq0        0.566557   0.065532    8.65  < 2e-16 ***
## revenue0    -0.000132   0.000135   -0.98     0.33    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## (Dispersion parameter for binomial family taken to be 1)
## 
##     Null deviance: 18228  on 16904  degrees of freedom
## Residual deviance: 11766  on 16894  degrees of freedom
## AIC: 11788
## 
## Number of Fisher Scoring iterations: 6

Model Evaluation

# Get the prediction probability for each data point
pred = predict(mRet,type="response")

# Set threshold as 0.5, and generate the confusion matrix
# TODO: What is a confusion matrix?
table(pred>0.5,CX$Retain)

##        
##         FALSE  TRUE
##   FALSE 12045  1530
##   TRUE    974  2356

# Compute accuracy
# TODO: What is accuracy?
table(pred>0.5,CX$Retain) %>% 
  {sum(diag(.))/sum(.)}

## [1] 0.8519

# Plot the ROC curve
# TODO: What is a ROC curve?
prediction(pred, CX$Retain) %>% 
  performance("tpr", "fpr") %>% 
  plot(print.cutoffs.at=seq(0,1,0.1))

# Compute AUROC (Area under ROC curve)
# TODO: What is AUROC?
colAUC(pred,CX$Retain)

##                  [,1]
## FALSE vs. TRUE 0.8792

Discussion

Observe the retention prediction results above:

Look at the coefficients of the logistic regression model. Which features are important for predicting retention? How do they affect the prediction result? Do you think the results make sense?
Besides the features used in this experiment, are there any other features you consider important for predicting retention?

Predicting Revenue

Now, we are going to learn a linear regression model for revenue prediction.

# We will only consider customers who made purchases
dx = subset(CX, Revenue > 0)

# Train a linear regression model, with y=log(Revenue) and X=recent + freq + log(1+money) + senior + status + freq0 + log(1+revenue0)
mRev = lm(log(Revenue) ~ recent + freq + log(1+money) + senior +
            status + freq0 + log(1+revenue0), dx)  

# Show the model training result
# TODO: What does R-squared represent?
summary(mRev)

## 
## Call:
## lm(formula = log(Revenue) ~ recent + freq + log(1 + money) + 
##     senior + status + freq0 + log(1 + revenue0), data = dx)
## 
## Residuals:
##    Min     1Q Median     3Q    Max 
## -3.245 -0.209 -0.067  0.205  3.435 
## 
## Coefficients:
##                    Estimate Std. Error t value Pr(>|t|)    
## (Intercept)        5.88e-02   4.58e-02    1.28   0.1997    
## recent             3.54e-04   5.07e-05    6.98  3.4e-12 ***
## freq               5.27e-02   4.65e-03   11.33  < 2e-16 ***
## log(1 + money)     9.32e-01   1.35e-02   68.94  < 2e-16 ***
## senior            -1.37e-04   1.82e-05   -7.52  7.0e-14 ***
## statusN2           1.28e-02   2.63e-02    0.49   0.6268    
## statusR1           1.93e-01   4.08e-02    4.73  2.3e-06 ***
## statusR2           2.98e-02   3.52e-02    0.84   0.3984    
## statusS1           8.24e-03   6.30e-02    0.13   0.8960    
## statusS2          -2.41e-01   8.66e-02   -2.78   0.0055 ** 
## statusS3          -3.67e-01   1.18e-01   -3.11   0.0019 ** 
## freq0              1.03e-02   1.73e-02    0.60   0.5501    
## log(1 + revenue0)  6.33e-02   9.40e-03    6.73  1.9e-11 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0.463 on 3873 degrees of freedom
## Multiple R-squared:  0.713,  Adjusted R-squared:  0.712 
## F-statistic:  802 on 12 and 3873 DF,  p-value: <2e-16

# Show the relation between predicted and ground truth revenue
plot(log(dx$Revenue), predict(mRev))
abline(0,1,col='red')

Discussion

Observe the revenue prediction results above:

Look at the coefficients of the linear regression model. Which features are important for predicting revenue? How do they affect the prediction result? Do you think the results make sense?
Besides the features used in this experiment, are there any other features you consider important for predicting revenue?

Prediction of Customer Lifetime Value

Predict Customer Retain And Revenue in 2016

Using data collected in 2015 to predict customer behavior in 2016

CX = Y$Y2015
names(CX)[8:9] = c("freq0","revenue0")

# Predict retention rate in 2016 using the model we trained earlier
CX$ProbRetain = predict(mRet,CX,type='response')

# Predict revenue in 2016 using the model we trained earlier
CX$PredRevenue = exp(predict(mRev,CX))

p0 = par(cex=0.8, mfrow=c(1,2))

# Draw the histogram for probability of Retain and Revenue
hist(CX$ProbRetain,main="ProbRetain")
hist(log(CX$PredRevenue,10),main="log(PredRevenue)")

par(p0)

CLV Estimation: Discounted Cash Flow (DCF)

\[DCF = \sum^N_{t=0}m\frac{r^t}{(1+d)^t}=m\sum^N_{t=0}(\frac{r}{1+d})^t \]

where $N$ denotes number of years under consideration, $m$ denoted marginal profit, $r$ is the retention rate, and $d$ is the discount rate.

# Compute DCF for 2015 data
N = 5     # Number of years under consideration = 5
d = 0.1   # Discount rate = 10%
CX$CLV = CX$PredRevenue * rowSums(sapply(
  0:N, function(i) (CX$ProbRetain/(1+d))^i ) )

# Show the stats of the estimated DCF
summary(CX$CLV)

##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
##       7      31      48     103      90   10188

# Show mean DCF for each group of customers
tapply(CX$CLV, CX$status, mean)

##     N1     N2     R1     R2     S1     S2     S3 
##  40.34 221.77 109.19 272.61  59.32  51.16  50.34

hist(log(CX$CLV,10))

Comparing The Customer Value Among Groups

# TODO: Show the mean predicted retention rate in 2016 of each group 
tapply(CX$ProbRetain, CX$status, mean)

##      N1      N2      R1      R2      S1      S2      S3 
## 0.20269 0.44075 0.34150 0.74925 0.05724 0.03475 0.02326

# Show the box plot of the estimates CLV of each group
boxplot(log(CLV)~status, CX)

# Show the violin plot of the estimates CLV of each group
ggplot(CX, aes(x=status, y=log(CLV), fill=status)) + 
  geom_violin(draw_quantiles=c(.25,.5,.75)) + 
  theme_minimal()

# TODO: Explain what this table shows
group_by(CX, status) %>% summarise(
  AvgExpRevenue = mean(PredRevenue),
  AvgRetainProb = mean(ProbRetain),
  AvgCLV = mean(CLV), 
  GroupSize = n() ) %>% 
  mutate(
    GroupValue = AvgCLV * GroupSize
  ) %>%  data.frame

Discussion

Observe the DCF estimation results above:

Do the estimated DCF values for each group reflect their behavior?
Compute and visualize the mean LTV and CLV for each group of customers. Can you observe a similar distribution between DCF and CLV/LTV?

Use the BG NBD model to compute CLV: https://retina.ai/academy/lesson/calculating-clv-with-r/

Selecting Marketing Target

Given the cost and expected revenue of a marketing tool, we are now deciding which group of customers to target with this tool.

Targeting `R2`

Visualize the predicted Retain and Revenue of R2

p0 = par(cex=0.8, mfrow=c(1,2))

# TODO: Show the histogram of probRetain and log_10(PredRevenue) of R2
hist(CX$ProbRetain[CX$status=="R2"],main="ProbRetain",xlab="")
hist(log(CX$PredRevenue[CX$status=="R2"],10),main="PredRevenue",xlab="")

par(p0)

Estimating Expected Return (ROI)

Assuming the cost and expected revenue of our selected marketing tool are

cost = 10        # Cost
effect = 0.75    # Effect: Probability of purchase in the next period

Estimate the expected return of this selected tool in R2

# Extract customer data of R2
Target = subset(CX, status=="R2")

# Compute the expected return
Target$ExpReturn = (effect - Target$ProbRetain) * Target$PredRevenue - cost

# TODO: Show the stats of ExpReturn
summary(Target$ExpReturn)

##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
##  -515.8   -15.4   -11.5   -10.3    -8.1   646.9

#Target = subset(CX, status=="R2")
Target

Choose The Marketing Target

# Sort the data by expected returns and trim down the data to include only "cid" and "ExpReturn"
Target = Target %>% arrange(desc(ExpReturn)) %>% 
  select(cid, ExpReturn)

# Compute the number of customers that have ExpReturn > 0
sum(Target$ExpReturn > 0)

## [1] 258

# TODO: Compute the sum of expected return from customers that have ExpReturn > 0
sum(Target$ExpReturn[Target$ExpReturn > 0])

## [1] 6464

If We Apply This Marketing Tool to Every Group

# Select all data
Target_all = CX

# TODO: Compute the expected return for all data
Target_all$ExpReturn = (effect - Target_all$ProbRetain) * Target_all$PredRevenue - cost

# Compute mean ROI and total ROI for each group, focusing on customers with ExpReturn > 0
filter(Target_all, Target_all$ExpReturn > 0) %>%
  group_by(status) %>% summarise(
    No.Target_all = n(),
    AvgROI = mean(ExpReturn),
    TotalROI = sum(ExpReturn) ) %>% data.frame

Discussion

Observe the ROI estimation results above:

Do the estimated ROI values for each group reflect their behavior?
If you were to modify this analysis process, what would you do to enhance the results?

Summary

If you only have three fields: customer ID, transaction date, and transaction amount, the analyses you can perform include:

For the entire customer base and each customer segment:

Group size and growth trends
Group attribute analysis: such as average CLV, average revenue contribution, growth rate, gross margin (cost data required), etc.
Inter-group traffic and average migration probability

For each customer:

Retention rate, expected purchase amount, lifetime value
Current group and the probability of transitioning to another group in the next period
If there are records of marketing tool usage, we can also estimate the success probability of each marketing tool for each customer

Generally, the results of these analyses are sufficient for us to formulate customer development and retention strategies; as for customer acquisition strategies, we usually need to extract personal attribute data from the CRM to achieve this.

Customer Value Prediction Example

Customer Value Management with Transaction Data

Introduction

Environment Setup

Exploring The Dataset

Loading The Dataset And Data Preprocessing

Customer Data

Data Visualization

Hierarchical Clustering

RFM (Recency, Frequency, Monetary) Clustering Using K-means

Discussion

Visualizing The Clusters

Discussion

Rule-based Segmentation

Customer Segmentation Rules

Customer Data in A Period

Visualize The Changes in Group Size

Dynamic Analysis of Features

Discussion

Customer Flows between Groups

Visualizing The Customer Flow

Discussion

Learning to Predict

Data Preprocessing

Predicting Retention

Model Evaluation

Discussion

Predicting Revenue

Discussion

Prediction of Customer Lifetime Value

Predict Customer Retain And Revenue in 2016

CLV Estimation: Discounted Cash Flow (DCF)

Comparing The Customer Value Among Groups

Discussion

Selecting Marketing Target

Targeting R2

Estimating Expected Return (ROI)

Choose The Marketing Target

If We Apply This Marketing Tool to Every Group

Discussion

Summary

Targeting `R2`